fix(analytics): detect AI agents from the variables that they set - #859

Requirements

• I have added test coverage for new or changed functionality
• I have followed the repository's pull request submission guidelines
• I have validated my changes against all supported platform versions

I ran the tests and the binary on Linux arm64 only.

Related issues

This PR fixes the agent detection that REL-12753 (#659) added.

Describe the solution you've provided

The agent_context event property tells us which AI coding agent ran ldcli. In the last 30 days, Mixpanel showed almost no named agents. Only one account sent claude-code, and only one user sent aider. All other agent runs went to unknown-non-interactive.

The cause is the list of environment variables in cmd/analytics/known_agents.json. Each name was compared to the source code, documentation, and live environment of the agent.

The first commit makes these changes:

1. It reads AI_AGENT before the other variables. AI_AGENT is a shared convention that agents.md#136 describes. Claude Code, Copilot in VS Code, Devin, OpenCode, Crush, and Amp set it. The gh CLI and @vercel/detect-agent read it. The value starts with the agent name, for example claude-code_2-1-295_agent. ldcli matches the start of the value. An unknown name becomes ai-agent:<name> without the version.
2. It replaces the list with variables that the agents set. It adds Gemini CLI, OpenCode, Kilo Code, Crush, Augment, Roo Code, Amp, and Amazon Q. Kilo Code also sets OPENCODE=1, so ldcli examines KILO first.
3. It lets an entry match a value with equals or contains. Amazon Q, Aider, and the Cursor terminal role need this.
4. It adds tests that use the variables that each agent sets. Other tests make sure that the old names and the variables that people set do not match.

The order of the checks stays the same: LD_CLI_AGENT, then the agent variables, then the TTY check, then the CI variables. Some Cursor versions set CI=1. Because ldcli examines CURSOR_AGENT before the CI variables, these runs now get the label cursor, not ci.

The second commit fixes a different problem in CLI Command Completed. Since #589 (version 1.16.2), Execute called os.Exit(1) before it sent the event. Thus ldcli did not send an error outcome. In the last 30 days, all 4,542 error events came from version 1.16.1 or earlier. Execute now sends the event before it exits. The exit code stays 1.

I ran both binaries in a Claude Code session against a local server that recorded the tracking requests:

main,   flags get with an error   CLI Command Run        agent_context=unknown-non-interactive
                                  (no CLI Command Completed event)
branch, flags get with an error   CLI Command Run        agent_context=claude-code
                                  CLI Command Completed  agent_context=claude-code outcome=error
main,   config --list             agent_context=unknown-non-interactive
branch, config --list             agent_context=claude-code

Describe alternatives you've considered

I did not use @vercel/detect-agent as a model. It detects Cursor with CURSOR_TRACE_ID and Copilot with COPILOT_MODEL. These variables also occur for people or come from the configuration of the user. I also did not use REPL_ID, TERM_PROGRAM, or the generic AGENT variable. Each of them also occurs when a person types the command.

Additional context

These limits remain:

1. Cline and Roo Code set their variables only in a VS Code terminal. In background mode, they do not set a variable.
2. Continue, Zed, Jules, and Warp do not set an agent variable.
3. The name WINDSURF_CASCADE_TERMINAL comes from the jfrog-cli detector. The Windsurf source code is not public.

The dashboard counts will change slowly, because only users who upgrade send the new labels. A blank agent_context does not always mean a person at a terminal. Versions before 3.0.0 do not send the property, and these versions sent about 45% of the blank runs.

🤖 Generated with Claude Code